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Abstract. — In this paper, we describe a process to create hyperbolicity in the neigh- 
bourhood of a homoclinic orbit to a partially hyperbolic torus for three degrees of freedom 
Hamiltonian systems: the transversality-torsion phenomenon. 

keywords: Hyperbolicity, Partially hyperbolic tori, Hamiltonian systems. 

1. Introduction 

The aim of this paper is to describe a process of creation of hyperbolicity in a partially 
hyperbolic context called the transversality-torsion phenomenon introduced in (jl],jS]). 
This process comes from the study of instability (Arnold diffusion) for (at least) three 
degrees of freedom near-integrable Hamiltonian systems pQ and more precisely from 
the derivation of a Smale-Birkhoff theorem (H^HU) for transversal homoclinic partially 
hyperbolic tori which come along multiple resonances |14j . Our starting point is the 
following conjecture of R.W. Easton ( 6 ,p.252) about symbolic dynamics for transversal 
homoclinic partially hyperbolic tori: In Easton prove the existence of symbolic 
dynamics in a neighbourhood of a partially hyperbolic torus whose stable and unstable 
manifolds intersect transversally (in a given energy manifold containing the torus). This 
result is obtained under hypothesis, the most stringent one being on the linear part of 
the homoclinic map (see §01 for a definition and |Sj,p.244), called the homoclinic matrix. 
However, Easton conjectures ([6, p. 252) that this assumption can be weakened, or perhaps 
cancelled. Moreover, the role of all the parameters of the problem (transversality, torsion 
of the flow around the torus) is not clear. 



In ( |3] , |S] ) ) we weaken the homoclinic matrix condition, but mainly, we put in evidence 
a dynamical and geometrical phenomenon at the origin of the hyperbolic nature of sym- 
bolic dynamics, the transversality-torsion phenomenon: the transversality of the stable 
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and unstable manifold of the torus coupled with the torsion of the flow around the torus 
give rise to a hyperbolic dynamics in the neighbourhood of the homoclinic connection. 
Since this first study, the transversality-torsion phenomenon has been identified and 
extended by others. We refer in particular to the papers of M. Gidea and C. Robinson ( 9 
p. 64) and M. Gidea and R. De La LLave jS], dealing with topological methods in dynamics. 

In this paper, we prove that the transversality-torsion phenomenon observed in a 
particular case in [5] arises in a generic situation for three degrees of freedom Hamiltonian 
systems. 

The plan of the paper is the following: In §|2 we define transversal homoclinic partially 
hyperbolic tori. In §|31 we state precisely the hyperbolicity problem, which can be resume as 
finding the minimal conditions (about the dynamics on the torus and the geometry of the 
intersection of the stable and unstable manifolds) in order to have a homoclinic transition 
map^ hyperbolic. In §0J we solve the hyperbolicity problem for three degrees of freedom 
Hamiltonian systems, putting in evidence the transversality-torsion phenomenon, i.e. the 
fundamental role of the torsion of the flow on the torus and the transversality of the stable 
and unstable manifolds to induce hyperbolicity of the transition map. 

2. Transversal homoclinic partially hyperbolic tori 

In this section, we define partially hyperbolic tori following the paper of S. Bolotin and 
D. Treschev 

2.1. Partially hyperbolic tori. — Let A4 be a 2m dimensional symplectic manifold, 
and H an analytic Hamiltonian defined on A4. 

Definition 1. — A weakly reducible, diophantine partially hyperbolic torus for H is a 
torus for which there exists an analytic symplectic coordinates system, such that the Hamil- 
tonian takes the form 

(1) H(6, 1, s, u) =uj.I+^ AI.I + s.M(9)u + 3 (/, s, u), 

where (6, 1, s, u) £ T fc xR fc xR m_fc xlR m_A: ; with the symplectic structure v = dlAdO+dsAdu, 
A is a k x k symmetric constant matrix, M is a definite positive matrix and for all 
k£Z n \ {0}, we have 

| uj.k \> a | k | _/3 , a,/3 > 0. 
If M is a constant matrix, then the partially hyperbolic torus is say to be reducible. 

^'See S I3.1l for a definition. 
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In (gl, Theorem 1, p. 406), S. Bolotin and D. Treschev prove that this "KAM" defi- 
nition is equivalent to the dynamical one (see [3J, Definition 1 and 3, p. 402). Moreover, 
for k = 1 and k = m — 1, the torus is always reducible. 

In |3J, Bolotin and Treschev introduce the notion of nondegenerate hyperbolic torus, 
which is a condition of dynamical nature (see [3], definition 3, p. 402). In the setting of 
weakly reducible hyperbolic tori, we can use the following definition which is equivalent 
to the dynamical one (see [3J, Proposition 2, p. 404): 

Definition 2. — A weakly reducible hyperbolic torus is nondegenerate if detA ^ 0. 

H. Eliasson [7] and L. Niederman |13| . have proved the following normal form theorem 
for m — 1 dimensional tori: 

Theorem 1. — Let T be a to — 1 dimensional reducible and non- degenerate diophantine 
partially hyperbolic torus. There exists an analytic coordinates system (x, y, z + ,z~) defined 
in a neighbourhood V of T, such that 

(2) H = uj.y + Xz'z + + 2 (y,z + z'). 

The geometry of the torus can then be easily described ( 3 ): it admits analytic stable 
(resp. unstable) manifold, denoted by W + (T) (resp. W(T)), and locally defined in V 
by: 

, . W + (T) = {(x,y, z + , z~) EV, y = 0, z~ = 0}, 

{ > W-(T) = {(x,y,z + ,z-) <=V, y = 0,z+ = 0}. 

2.2. Transversal homoclinic connection. — In the following, we denote by H the 
energy submanifold of M. containing the torus under consideration. For convenience, a 
weakly reducible diophantine partially hyperbolic torus will be called a partially hyperbolic 
torus. 

Definition 3. — Let T be a to — 1 dimensional partially hyperbolic torus. We say that T 
possesses a transversal homoclinic connection if its stable and unstable manifolds intersect 
transver sally in 7i. 

In this paper, we explore the existence of a hyperbolic dynamics in a neighbourhood of 
a transversal homoclinic connection to a partially hyperbolic torus. 



3. The hyperbolicity problem 

3.1. Set-up. — Let H be a to degree of freedom Hamiltonian system. Let T be a 
to — 1 dimensional partially hyperbolic torus of H possessing a transversal homoclinic 
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connection along a homoclinic (at least one) orbit denoted by 7. We introduce the 
following notations and terminology: 

Let V be the Eliasson's normal form domain (J2J). There exists (_11J, a Poincare section 
S of T in V, and an analytic coordinates systems in S, denoted by ((f), p, s, u) G ir m ~ 2 x 
K x R m_2 x K, such that the Poincare map takes the form 

(4) /O, s, p, u) = (cj) + lo + i/p, As, p, A _1 u) + 2 (p, s, u), 

where u G M m ~ 2 , v G M" 1 " 2 , < A < 1, vp = (v x px, v m _ 2 Pm-2)- 

We denote by fi(4>, s,p,u) = (4> + uj + i/p, As, p, A _1 u) the linear part of /. 

We say that the torus T is with torsion if 7^ 0, for i = 1, ... ,m — 2, and without 
torsion otherwise. We note that a torus is with torsion if and only if it is nondegenerate. 

Let p~ = 0, 0, u~) G S and p + = (0 + ,s + ,0, 0) G 5, be the last (resp. the first) 
point of intersection between 7 and S along the unstable manifold (resp. the stable 
manifold). There exists neighbourhoods V + and V~ in S of p + and p~ respectively, and 
a map T : V~ — > y + , called the homoclinic map, such that T(p _ ) = p + . The homoclinic 
map is of the form 

T{p- + z) =p + + U.z + 2 (z), 
where II is a matrix, called the homoclinic matrix. We denote by 

Ti(p~ + z) =p + + IL.z. 

We denote by D n = {z G V + \ ff(z) G V~} and D = U n >i ^n- We denote by 
ip : D — s- V~, the transverse map introduced by Jiirgen Moser |12j and defined by 

%b(z) = f\z) if z G D n . 

We denote by ipi(z) = ff(z) if z G D n . 



The differential of //, denoted by Dfi is the matrix 
(5) Dfx 



( Id V \ 
A 



Id 
\ A -1 J 

where Id is the (m — 2) x (m — 2) identity matrix and V the diagonal matrix with 
components z/j, i = 1, . . . , m — 2. 
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In the following, we always work in the Poincare section S. 

Let C = {(u,v) G R m_1 x M m_1 ||| u ||i< 1, || v ||i< 1}. We denote by W„ : C -► V+, 
what we call an Easton's window (or simply window in the following) defined by 

Wftiz) = fiz + p + . 

We consider the map A : C — > C, defined by 

A = (^r 1 oro^oif /1 . 

We denote by 

a, = (w^y 1 or (0 ^o w M . 

We have A and A; as close as we want in C 1 -topology when \i — > ([Bj,p.250). 
The map r o ip is called the homoclinic transition map. 

In the following, we call linear model a Hamiltonian system possessing a transversal 
homoclinic partially hyperbolic torus T such that the preceding maps are linear in a given 
coordinates systems. Using the Easton's window Wu, the general case reduces to prove 
results on the linear model which are stable under small C 1 perturbations. This is the 
case for example when dealing with symbolic dynamics. 

3.2. The hyperbolicity problem. — We keep the notations and terminology of the 
previous section. For all matrix M, we denote by spec(M) its spectrum. The hyperbolicity 
problem can be formulated as follow: 

Hyperbolicity Problem - Let H be am degrees of freedom Hamiltonian system. Let 
T be a m — 1 dimensional partially hyperbolic of H possessing a transversal homoclinic 
connection. Under which conditions on n, v and II do we have 

spec(n.^/; n ) n 5 1 = 0, 

where S 1 = {z € C, | z \= 1} is the unit circle in C. 

If spec(n.Z)/™) n S 1 = 0, then for fj, sufficiently small, i.e. in a given neighbourhood of 
the homoclinic orbit, the map A is hyperbolic. 
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This problem is difficult as there exists no results about localization of eigenvalues for 
the product of two matrices^ 2 ) . In the following, we solve the hyperbolicity problem in the 
three degrees of freedom case. 



4. The transversality-torsion phenomenon 

In this section, we deal with three degrees of freedom Hamiltonian systems. In the 
following, we denote by A4 np (M.) the set of n x p matrices with real coefficients and for all 
matrices M € .A/f nX n(R)> we denote by | M | its determinant. 

4.1. Transversality constraints. — The matrix II G Ad^QH) has the following form 
in the symplectic base (e<^, e s , e p , e u ): 

u=( A B 

\C D 
where A,B,C,D € M 2 , 2 (R). 

For all differentiate manifold A4, we denote by T X M. the tangent space to M at point 
x € M. 

Definition 4- - - We say that the homoclinic matrix is transverse if and only if it satisfies 
the following transversality condition U(T p -W~ (T)) + T p +W + (T) = T p +S. 

Of course, if the intersection of the stable and unstable manifold, W + (T) and W~(T), 
of a torus T is transverse along an homoclinic orbit 7, then the homoclinic matrix satisfies 
the transversality conditions by definition. 



Lemma 1 . — The matrix U is transverse if and only if A 



ci,i di,2 
C2,l d 2 ? 



^0. 



Proof. — Let v = [v^, 0, 0,v u ) be a vector in T p -W (T). We have 

(6) Uv = (auv^ + b 12 v u , a 2 iv (/ , + b 2 2V u , cn^ + di 2 v u , c 2 iv^ + d 22 v u ). 

We begins with the global condition of transversality, namely that v' = Uv = (v'^, v' s ,v' p , v' u ) 
is such that v' p = and v' u = if and only if v<j, = and v u = 0. This condition implies 



ci,i di, 2 
C2,l d 22 



^0. 

□ 

In the following, we need the following strengthening of the transversality condition: 



There exists hyperbolicity results for random or deterministic product of matrices like [2]. However, 
they are based on genericity arguments which can not be used in order to understand the role of each of 
the elements n, v and IT in the creation of hyperbolicity. 
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Definition 5. — The matrix U is strongly transverse if A ^ and c?2,2 7^ 0. 

The condition cZ2,2 7^ does not come from the transversality assumption. We can 
understand the geometrical nature of this condition as follow: 

The unstable (resp. stable) manifold W U (T) (resp. W S (T)) is foliated by 1 dimensional 
manifolds (see [15] .n. 138) denoted by W£{T) (resp. W°{T)), p£T (the Fenichel fibers), 
and 

(7) W U (T) = (J W?(T) (resp. W S (T) = (J W;(T) ). 

In the normal form coordinates system, we have for all p = (4> p , 0, 0, 0) 6 T, 

(8) ^,0,0,0) ( T ) = s,p,u) € T x R x R x K | = p , s = 0, p = 0}, 

(9) %o,o,o)( T ) = {(^ s .ft tl ) eTxRxRxR </> = p , u = 0, p = 0}. 

The condition c^2,2 7^ is then equivalent to the following geometrical condition on the 
foliation of the stable and unstable manifolds in the linear model. 

Lemma 2. — Let us consider the linear model. The condition d>2,2 ^ is equivalent to 
the transversality of the intersection between the unstable leave at ((f)—, 0,0,0) denoted by 
Wfy Q \(T) with the invariant manifold defined by {(</>, s, p,u) £TxIxlxl;u=0} 
at point (4> + , 0, 0). 

4.2. The transversality-torsion phenomenon. — The main technical result of this 
paper is the following: 

Theorem 2 (Transversality-torsion phenomenon). — Let H be a 3 degrees of 
freedom Hamiltonian system possessing a 2 dimensional partially hyperbolic tori with a 
transversal homoclinic connection. We keep notations from section^ We assume that: 

i) The homoclinic matrix U is transverse; 

ii) The torus is with torsion; 

Then, for n sufficiently large, the matrix ILD/, n is hyperbolic. 

Moreover, if the matrix II is strongly transverse, i. e. c?22 7^ 0, all its eigenvalues are 
reals and given asymptotically by 

X\ ~ —ni/d^ A, X2 ~ ^22^"™, X3 = X^ 1 , X4 = X% . 

Proof. — Let us assume that the matrix UDfp possesses a complex eigenvalue (3. As 
HDf™ is symplectic, we know that the three remanning eigenvalues are (3, 1//3 and 1/(3 
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(see jlOj . prop. 5.5.6, p. 220). The characteristic polynomial is then given by 



P n (x) = x 4 + A(n)x 3 + B(n)x 2 + A(n)x + 1, 




(S + S), B(n) = 2+ | S | 2 with S = (3 + 



1 



-d 2 2A n - A n a 2 2 - nvc\\ - an - dn, 

X n [\ A | +a 2 2<iii - C12&21 + nv(a,22C\\ - 012021)] 



+A 71 [| D I +and 2 2 - C21612 + nuA] 
+(aiidn + 022^22 - C22&22 - C11611). 



We must consider two cases: (I22 7^ and 0^2 = 0. 

- If (I22 7^ 0, i.e. we have for n sufficiently large A{n) ~ — d,22^~ n - In the same way, 
as A / and v 7^ 0, we obtain B(n) ~ nvA\~ n . We deduce that ReS ~ c^A - ™ and 
I ^ | 2 ~ ^A 2 ™- We also have | S | 2 ~ nvA\~ n using the inequality on B(n). We obtain 
a contradiction. As a consequence, all the eigenvalues are reals. 

We then have eigenvalues x\, X2 and 1/xi, I/X2, 2:1 € R and £2 G K. We denote by 
5i = xi + 1/xi and S2 = £2 + l/a^- We have A(n) = — (Si + S2) and i?(n) = 2 + /Si 52, 
so Si(A(n) + Si) = —SiS2- As A(n) ~ — c^A - ™ and -B(n) ~ nz^AA _n , we conclude that 
Si ~ — nd^A, so xi ~ — nd^A. Using A(n), we obtain S2 ~ cfeA - ™, so X2 ~ d22^~ n , 
which concludes the proof. 

- If c?22 = 0, we have A(n) = 0(n). As B(n) ~ ni , AX~ n , this implies that Im(S) 7^ 0. 
If we denote = fa+ifo, ^faS K, we have Im(S) = /? 2 (1- | /? As Im(S) ^ 0, we 
deduce that /3 2 7^ and 1— | f3 0, i.e. /?2 7^ and | [5 |/ 1. The eigenvalues are then 
hyperbolic. 

This concludes the proof. □ 

The behaviour of the eigenvalues can be also given when ^22 = 0, but depends on several 
assumptions on the form of the homoclinic matrix which have not a direct geometrical 
meaning. 

In some cases of interest, we can obtain a stronger result. For example, using the 
homoclinic matrix introduced in jllj and generalized in [4| in relation with the Arnold 
model pQ, we obtain: 



Theorem 3. — Let H be a three degrees of freedom Hamiltonian systems possessing a 2 
dimensional partially hyperbolic tori with a transversal homoclinic connection. We keep 
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the notations o/§[3J We assume that the homoclinic matrix has the form 

/ 1 \ 

(io) n= 0100 

{ U) 5 10 

\ 1 J 

and 5 is a parameter. 

Then, the matrix HDfJ 1 is hyperbolic for n sufficiently large if and only if the matrix II 
is transverse, i.e. 5^0 and the torus is with torsion, i.e. v ^ 0. 

Proof. — The characteristic polynomial of HDfJ 1 is given by 

P(x) = (x 2 - x(5nu + 2) + l)(x 2 - xa(n) + 1), 

where a{n) = X 2n + A~ n . The matrix is hyperbolic if 5 ^ and v ^ 0. Indeed, in this 
case, the matrix II satisfies the transversality assumption. Moreover, if 5 = and v ^ 
(or 6^0 and v = 0), we obtain two eigenvalues equal to ±1, destroying the hyperbolicity. 
This concludes the proof. □ 
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